
The Journal of Effective Teaching 

an online journal devoted to teaching excellence 


The Use of a Checklist and Qualitative Notebooks 
for an Interactive Process of Teaching and Learning 
Qualitative Research 

Rebecca K. Frels la , Bipin Sharma b , Anthony J. Onwuegbuzie c , 

Nancy L. Leech d , Marcella D. Stark c 
A Lamar University, Beaumont, Texas 77710 
b Saint Xavier University, Chicago, IL 60655 
c Sam Houston State University, Huntsville, Texas 77341 
d Un iversity of Colorado Denver, Denver, Colorado 80217 


Abstract 

From the perspective of doctoral students and instructors, we explain a developmental, 
interactive process based upon the Checklist for Qualitative Data Collection, Data Analy¬ 
sis, and Data Interpretation (Onwuegbuzie, 2010) for students’ writing assignments re¬ 
garding: (a) the application of conceptual knowledge for collecting, analyzing, and inter¬ 
preting qualitative data; (b) utilizing detailed instructor feedback of content and writing 
style as a positive catalyst for growth; and (c) integrating feedback in future assignments 
and reflecting on the process. We advocate the cyclical use of this framework for teach¬ 
ing and learning rigorous qualitative research. 

Keywords: Writing, writing rubric, qualitative research, steps in qualitative re¬ 
search process. 


Graduate students enrolled in qualitative research courses often undertake multiple daunt¬ 
ing tasks in order to understand and to conduct rigorous qualitative research. These tasks 
include learning: (a) the history and current state of qualitative inquiry, (b) numerous 
qualitative research designs, and (c) methods of analyzing and interpreting qualitative 
data. A few common textbooks offer traditions (e.g., phenomenological research, 
grounded theory, ethnography) and methods for conducting rigorous qualitative studies 
(cf. Chenail, 2007; Denzin & Lincoln, 2005; Hurworth, 2008; Miles & Huberman, 1994). 
However, to leam how to conduct rigorous qualitative studies, more than textbook infor¬ 
mation is necessary. Moving from reading theory in the texts to applying the concepts 
through practice involves an application of concepts that is often reliant upon student 
writing and instructor feedback. 

In order for any research study to be shared with others, with some exceptions (e.g., per¬ 
formance ethnography, visual anthropology), the study results are in written form. In- 
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deed, writing well is a mainstay for productive scholarship for students and academics 
alike, and the phrase publish or perish might be all too familiar to even prolific writers. 
According to Boote and Beile (2005), “acquiring the skills and knowledge required to be 
education scholars should be the focal, integrative activity of predissertation doctoral 
education. Preparing students to analyze and synthesize research in a field of specializa¬ 
tion is crucial to understanding educational ideas” (p. 3). Thus, in this article, to assist 
researchers in their ability to write qualitative research, doctoral students and instructors 
describe the interactive process of Qualitative Notebooks (Onwuegbuzie et ah, 2009) and 
the Checklist for Data Collection, Data Analysis, and Data Interpretation, developed by 
Onwuegbuzie (2010), hereafter called the Checklist. This instrument, which facilitates 
the development of becoming a prolific qualitative researcher, is used with a writing 
component in qualitative research coursework. Even though the Checklist was created for 
the development of students as qualitative researchers, we believe that the Checklist is a 
valuable tool to confirm the essential elements of a qualitative study for emergent schol¬ 
ars and beyond. To evidence the developmental, interactive process of the Checklist as an 
integral writing component for rigorous qualitative research and to evidence the interac¬ 
tive process of the instructor and student(s) through the Checklist and the Qualitative 
Notebook(s), we present following sections: (a) Theoretical Framework; (b) Methodo¬ 
logical Framework; (c) Model for Teaching and Teaming Qualitative Research, (d) The 
Qualitative Notebook; (e) Checklist for Data Collection, Data Analysis and Data Interpre¬ 
tation; (f) APA Style: Part II of the Checklist; (g) The Interactive Writing and Feedback 
Process; (h) The Reflexive Journal; (i) Reflections on the Process, and (j) Implications. 

Theoretical Framework 

According to Vygotsky (1978), learning encompasses a scaffolding effect through the 
apprenticeship process. Hence, Vygotsky’s (1978) theory of social development under¬ 
scores the developmental feedback cycle associated with the interaction between the pro¬ 
fessors) and student(s) and the use of the Checklist. As such, the process of learning 
qualitative inquiry is an interactive process whereby instructors interact with students in 
both verbal and written format, stressing each important stage of student development 
and students’ belief systems regarding their qualitative research. 

Methodological Framework 

In addition to the theoretical research framework, Feech and Onwuegbuzie’s (2010) in¬ 
teractive, iterative, recursive, emerging, and continuous 13-step process for qualitative 
research was used as the methodological framework for teaching and learning qualitative 
research: (a) Step 1: Detennine the goal of the study, (b) Step 2: Formulate the research 
objective(s), (c) Step 3: Determine the rationale of study, (d) Step 4: Determine the re¬ 
search purpose, (e) Step 5: Determine the research question(s), (f) Step 6: Select the 
qualitative sampling framework, (g) Step 7: Select the qualitative research design, (h) 
Step 8: Collect data, (i) Step 9: Analyze data, (j) Step 10: Fegitimate data, (k) Step 11: 
Interpret data, (1) Step 12: Write the qualitative research report, and (m) Step 13: Refor¬ 
mulate the research question(s). As such, students use the methodological framework as a 
structural component when designing and undertaking qualitative research studies. 
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Model for Teaching and Learning Qualitative Research 

The model for teaching and learning qualitative research (Onwuegbuzie et al., 2009) al¬ 
lows students to progress through four major phases of learning and practice. The first 
phase, the conceptual/theoretical phase, is an introduction to the qualitative research 
process. In this phase, students explore the history of qualitative research, as they also 
recognize the first five steps (Leech & Onwuegbuzie, 2010), which are involved in re¬ 
search formulation, specifically: (a) Step 1 (detennine the goal), (b) Step 2 (formulate the 
research objective), (c) Step 3 (determine the rationale), (d) Step 4 (determine the re¬ 
search purpose), and (e) Step 5 (determine the research question). 

In the second phase of the course, the technical phase, instructors describe 18 qualitative 
analysis techniques (i.e., classical content analysis, constant comparison analysis, and 
discourse analysis) using qualitative data analysis software (NVivo 8; QSR International 
Pty Ltd., 2008). In addition, students compose a research plan (i.e., Step 6: select the 
sampling framework and Step 7: select the research design). Also, research implementa¬ 
tion (i.e., Step 8: collect data, Step 9: analyze data, Step 10: legitimate data, and Step 11: 
interpret data) and disseminating results/suggesting future research (i.e., Step 12: write 
the research report and Step 13: reformulate research questions) frame the research proc¬ 
ess for students as they use the Checklist for writing research reports. The interactive 
process of teaching and learning culminates during the third phase of the course, the ap¬ 
plied phase, whereby instructors work closely with students in collecting, analyzing, in¬ 
terpreting, and writing their respective qualitative research assignments through the inter¬ 
active feedback process. Phase 4, the emergent scholar phase, is an extension of the third 
phase and occurs at the conclusion of the course. In this phase and beyond, instructors 
encourage students to present collaborative research and submit post-course manuscripts 
to journals for consideration for publication. Furthermore, as a framework for future 
qualitative studies, students are able to continue the use of the Checklist to guide them in 
data collection, data analysis, and data interpretation in conjunction with the 13-step 
process for qualitative research. 

Thus, the course phases are distinct, overlapping, and iterative. Students progress in their 
qualitative research knowledge and writing proficiency in Phase 2: the technical phase. 
In addition to learning 18 qualitative analysis techniques (Phase 2), students apply their 
knowledge and write six qualitative analysis exemplar notebooks (i.e., assignments) to¬ 
ward becoming emergent scholars (Phase 4). With this in mind, the writing component of 
the course is structured so that students: (a) author qualitative notebooks using the Check¬ 
list, (b) receive detailed instructor feedback via the Checklist and accept feedback as 
helpful toward setting goals, and (c) integrate feedback in future qualitative notebook as¬ 
signments and reflect upon the process. The following section, The Qualitative Notebook, 
explains how students build on the theoretical and conceptual knowledge of Phase 1 of 
the qualitative research course through the practical application of conducting research 
and writing research reports. 
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The Qualitative Notebook 

With respect to the assignment aspect of a course, there is a distinct difference between 
authentic assessments and performance assessments of coursework, with the former in¬ 
volving the application of knowledge in an authentic setting (Linn & Gronlund, 1999). 
Furthermore, in authentic assessments, authenticity is required, as opposed to perform¬ 
ance assessments whereby authenticity is only approximated (Linn & Gronlund, 1999). It 
is important to note that in order for students to engage in authentic assessments, they 
must undertake a practical application of concepts learned in coursework. Onwuegbuzie 
and Leech (2003) described the use of notebooks and detailed scoring checklists (e.g., 
scoring rubrics) to encourage strongly the development of students’ practical skills. Thus, 
in Phase 3 of the course (the applied phase), students are asked to become researchers. 

The qualitative notebook (e.g., authentic assessment) is one of the primary activities 
within the course for students to experience conducting qualitative research. The note¬ 
book assignment represents a series of assessments whereby students are asked to collect, 
to analyze, and to interpret data, and ultimately to write up formally the findings and in¬ 
terpretations in the same manner as would appear in a published journal article. In fact, 
the term notebook represents the idea of compiling each assignment that contains detailed 
feedback from instructors into a portfolio (i.e., notebook). Thus, students refer to their 
knowledge gained in Step 1 through Step 5 for formulating their research, and move into 
Step 6 (i.e., select sampling framework) and Step 7 (i.e., select research design) as they 
compose the method sections of their research studies to be presented through the note¬ 
book assignments. It is at this point that students use the Checklist to understand the nec¬ 
essary components regarding data collection, data analysis, and data interpretation to in¬ 
clude when writing their notebooks. Figure 1 depicts the Checklist for use within the 13- 
step methodological framework (Leech & Onwuegbuzie, 2010), used in the structure of 
the course with the four-phase model for teaching and learning qualitative research (On¬ 
wuegbuzie et ah, 2009). As seen in Figure 1, the four-phase model allows students to 
bridge the conceptual and theoretical components (e.g., the 13-step methodological 
framework) of qualitative research into the writing process. In addition, depicted in Fig¬ 
ure 1, students refer to specific steps of the 13-step qualitative research process at differ¬ 
ent times of the course. For example, students refer to Step 8 and Step 9 (research imple¬ 
mentation) in the Results section, and Step 12 and Step 13 (disseminate results/suggest 
future research) in the Discussion section. As presented in this figure, the Checklist is in¬ 
troduced in Phase 2 (i.e., the technical phase). 

In addition, students use software (NVivo 8; QSR International Pty Ltd., 2008) to analyze 
data extracted from interviews, member checking interviews, and debriefing interviews 
for a case study for six assigned notebooks: (a) word count/keywords-in-context; (b) clas¬ 
sical content analysis; (c) method of constant comparison; (d) ethnographic analysis (i.e., 
domain analysis, taxonomic analysis, componential analysis); (e) discourse analysis; and 
(f) cross-case displays: exploring and describing/ordering and explaining. Specifically, 
students use the Checklist and detailed feedback from instructors to help guide their sub¬ 
sequent qualitative notebook write-ups and, as a result, the quality of the write-ups poten¬ 
tially improves for students with each subsequent qualitative notebook report, until each 
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Phase One: 

Conceptual/ 

Theoretical 

Phase 



Research Formulation 

Step 1: Determine Goal 

Step 2: Formulate Research Objective 

Step 3: Determine Rationale 

Step 4: Determine Research Purpose 

Step 5: Determine Research Question 



Components of the Checklist for 
Data Collection, Data Analysis, 
and Data Interpretation 


Research Plan 

Step 6: Select Sampling Framework 
Step 7: Select Research Design 



Research Implementation 

Step 8: Collect Data 
Step 9: Analyze Data 


(Title) 

1. Method 

Participants 

Instruments 

Procedure 

Legitimation 

Analysis 

2. Results 



I 

Research Implementation 

Step 10: Legitimate Data 
Step 11: Interpret Data 

Disseminate Results/Suggest Future Research 

Step 12: Write Research Report 

Step 13: Reformulate Research Questions 


3. Discussion 


4. Reference List 


5. Appendix 


Figure 1. Integration of the four-phase model, the checklist, and the 13-step meth¬ 
odological framework with respect to writing a research report. 
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respective notebook reaches maximum quality. In the following section, we describe how 
the Checklist is used to facilitate the interactive process between instructors and students 
for composing notebook assignments. 

Checklist for Data Collection, Data Analysis, and Data Integration 

Students compose respective qualitative notebooks according to two sections of the 
Checklist: (a) content (Part I), the rubric for instructors for feedback pertaining to the es¬ 
sential components of qualitative research; and (b) style (Part II), the rubric for feedback 
pertaining to adherence to American Psychological Association (APA, 2010) guidelines. 
Specifically, Part I contains 158 items with a Likert-format scale format for each item 
ranging from 1 to 5 (1 representing strongly disagree to 5 representing strongly agree). 
Likewise, Part II (i.e., writing style) contains 70 items. For Part II, as a guide, students 
are given an evidence-based article wherein the 60 most common APA errors are identi¬ 
fied (see Onwuegbuzie, Combs, Slate, & Frels, 2010). Both summative and formative 
feedback are given with the Checklist. Scores from both rubrics (i.e., content and style) 
are converted into percentages for a final score and summative feedback: 60% of the final 
score is for content (Part I) and 40% of the final score is for writing style (Part II). Thus, 
each qualitative notebook receives a score on a 100-point scale. Moreover, the tracking 
feature of the Word processing software program is used to provide formative feedback 
in conjunction with the Checklist. Formative feedback allows students to take remedial 
measures where necessary and for instructors to reinforce positive behavior, as waiting 
until the end of a course does not improve skills within the same course (Clarke, 1985). 
Hence, students are very aware of the expectations for a final assignment in the course 
and the various components and skills that are essential for rigorous research. When a 
student uses Part I as a guide for writing an exemplar notebook and discovers that he or 
she has neglected a critical element for presenting research, the student then seeks out 
through reference books ways to address essential components for reporting research 
findings. Part I of the Checklist pertains to 10 essential sections of a research report: (a) 
title, (b) method, (c) instruments, (d) procedure, (e) analysis, (f) legitimation, (g) results, 
(h) discussion, (i) reference list, and (j) appendix. Appendix A presents the Checklist 
Part I that lists each of the 158 items included in the 10 essential sections. It should be 
noted that this Checklist is continually revised and updated as new qualitative concepts, 
theories, procedures, and language emerge. Each of the essential sections is described in 
detail. 

Title 

According to APA (2010) style guidelines, a title should be “fully explanatory when 
standing alone” (p. 23) and summarize the main idea of a manuscript. The title section of 
Part I includes five pertinent points for students to consider when composing a title. With 
this in mind, specific components of the title section include directives to: (a) make clear 
the population/context/case(s) of interest; (b) make clear the primary independent vari¬ 
able^), if applicable; (c) make clear the dependent variable, if applicable; (d) indicate 


Appendix A is posted online at http://www.uncw.edu/cte/et/articles/Voll 1 1/FrelsAppA.pdf . 
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the specific relationship between the major variables; and (e) avoid vague, ambiguous, 
and emotional-laden terms. Thus, the title of a qualitative study, with regard to the five 
elements aforementioned, would depend on whether the qualitative inquiry is case- 
oriented, variable-oriented, or process/experience-oriented (Onwuegbuzie, Slate, Leech, 
& Collins, 2009). 

Method 

The method section of qualitative research involves Step 6 (select the sampling frame¬ 
work) and Step 7 (select the research design; Leech & Onwuegbuzie, 2010). As such, the 
method section should be detailed in explanation whereby researchers emphasize an ex¬ 
perience that is transformed into words for understanding “ naturally occurring, ordinary 
events in natural settings, so that we have a strong handle on what ‘real life’ is like” 
(Miles & Hubennan, 1994, p. 10, emphasis in original). Critical elements of qualitative 
research regarding the method section of Part I include: (a) participants, (b) instruments, 
(c) procedure, (d) legitimation, and (e) analysis. Each of these is described in detail. 

Participants. Sampling is an essential step of qualitative research, taking into account 
both: (a) the sampling scheme (i.e., the specific technique utilized to select participants); 
and (b) the sampling design (i.e., the framework within which the sampling occurs, com¬ 
prising the number and types of sampling schemes and the sample size; Onwuegbuzie & 
Leech, 2007a). Further, Onwuegbuzie and Leech (2007b) described the use of qualitative 
power analysis to represent “the ability or capacity to perform or act effectively with re¬ 
spect to sample” (p. 117). With regard to participants, qualitative power analysis assesses 
the appropriateness of units in a sample (Onwuegbuzie & Leech, 2007b). Additionally, 
Onwuegbuzie and Leech (2005) affirmed that qualitative researchers should make sam¬ 
pling decisions regarding the number of interviews, observations, focus groups, and 
length of interviews based upon the goal of attaining prolonged engagements, persistent 
observations, and adequate reflection (i.e., reflexivity). The participants section of Part I 
includes the aforementioned components regarding sampling and 16 major points pertain¬ 
ing to participants and the role of the researcher. Hence, students who thoroughly address 
each component of the method section should be confident that a study is well described 
for an audience. 

Instruments. Although researchers are the primary instrument in qualitative studies 
(Lincoln & Guba, 1985; Paisley & Reeves, 2001), most qualitative research studies in¬ 
volve the utilization of other instruments such as interview schedules, focus group sched¬ 
ules, and observational protocol. As recommended by Onwuegbuzie and Leech (2007a), 
a researcher should leave an audit trail by documenting every step of the research proc¬ 
ess, including activities, interviews, member-checks, and analysis of data. By employing 
the instrument section of Part I as a guide, students address 28 areas important for de¬ 
scribing instruments in qualitative research. Further, students are requested to provide 
citations for all statements pertaining to the characteristics of the instruments. Therefore, 
students leam important qualitative concepts such as non-verbal nuances (Fontana & 
Frey, 2005) regarding qualitative interviews, specifically: (a) proxemics, the use of inter¬ 
personal space to communicate ideas; (b) chronemics, the way speech and silence is con- 
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veyed through conversation; (c) kinesics, body movements or postures; and (d) paralin¬ 
guistics, the variations in volume, pitch, and quality of voice. 

Procedure. The compilation of research records should entail a complete ethnographic 
record, as suggested by Spradley (1980), to include all field notes, artifacts, and any other 
documentation relevant for understanding a qualitative research study. Furthermore, this 
section assists students to understand the importance of providing evidence for rich data, 
and systematic collection of data. Included in the 27 items of the procedure section is the 
task for students to identify at least one verification procedure in detail (e.g., prolonged 
engagement, persistent observation, triangulation, contextualization of observations, 
method of constant comparison, and examining extreme cases). 

Analysis. The analysis section (Step 9; Leech & Onwuegbuzie, 2010) of qualitative re¬ 
search involves the idea that using different qualitative data analysis techniques can in¬ 
crease triangulation; thus, researchers should systematically select multiple appropriate 
analysis techniques (Leech & Onwuegbuzie, 2008). Miles and Huberman (1994) con¬ 
tended that strengths of qualitative data depend upon their “ richness and holism, with 
strong potential for revealing complexity; such data provide ‘thick descriptions’ that are 
vivid, nested in a real context, and have a ring of truth that has strong impact on the 
reader” (p. 10, emphasis in original). Hence, the analysis section of the Checklist com¬ 
prises four general categories of analyses as noted by Leech and Onwuegbuzie (2010): 
(a) inductive coding (i.e., going to the text, coding inductively for meaning, developing 
categories from codes, producing theme statements); (b) deductive coding (i.e., coding 
deductively for meaning, going to the text, developing categories from codes, producing 
theme statements); (c) abductive coding (i.e., coming from an interactive process of de¬ 
duction and induction; and (d) interpretive coding (i.e., developing a more abstract inter¬ 
pretation). As such, qualitative data analysis consists of three concurrent activities: data 
reduction, data display, and conclusion drawing/verification (Miles & Huberman, 1994). 

Students are guided to make pre-analysis considerations important for addressing legiti¬ 
mation to include prolonged engagement, persistent observation, triangulation, leaving an 
audit trail, member checking, using extreme cases, replicating a finding, and referential 
adequacy (Leech & Onwuegbuzie, 2010). Importantly, and noted in the Checklist, stu¬ 
dents are asked to justify categories that emerge, and at which point (i.e., a priori, a poste¬ 
riori, or iterative) these categories were specified (Onwuegbuzie, 2003). Furthennore, 
students seek to understand the difference between exploratory or confirmatory tech¬ 
niques, the importance of naming sources used to identify categories, and ways to justify 
the existence of a given set of categories (i.e., external, rational, referential, empirical, 
technical, or participative; Constas, 1992). 

Legitimation. Denzin and Lincoln (2005) attributed three crises (representation, legiti¬ 
mation, and praxis) as confronting researchers regarding the evaluation of qualitative re¬ 
search and posed “a serious rethinking of such tenns as validity, generalization, and reli¬ 
ability' (p. 19; emphasis in original). Threats to internal and external credibility occur at 
three major stages of the research process, namely the research design/data collection 
stage, the data analysis stage, and the data interpretation stage (Onwuegbuzie & Leech, 
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2007a). Based on Onwuegbuzie and Leech’s (2007a) framework, the legitimation section 
of the Checklist assists students in addressing threats to internal credibility and external 
credibility in an iterative fashion. Using the Checklist, students detail, describe, and pro¬ 
vide the citations for all important threats to legitimation and verification procedures. The 
legitimation section of Part I includes references to guide students through potential 
threats to credibility (e.g., Creswell, 2007; Guba & Lincoln, 2005; Lather, 1991; Lincoln 
& Guba, 1985; Miles & Huberman, 1994). 

Authenticity criteria (Guba & Lincoln, 2005) embody constructivist assumptions for five 
elements that help to expand understanding and increase legitimation: (a) fairness: the 
researcher’s ability to value and honor the evaluation process; (b) ontological authentic¬ 
ity: the criteria for assessing a high level of awareness among participants in research; (c) 
educative authenticity: the extent to which participants understand and appreciate diverse 
value systems of others; (d) catalytic authenticity: the appreciations and constructions 
that lead to actions or decisions by participants; and (e) tactical authenticity: the empow¬ 
erment on participants and stakeholders to act on increased understanding that results 
from a study. As posited by Onwuegbuzie, Leech, and Collins (2008), the ability to dis¬ 
cuss thoughts, perceptions, feelings, and experiences regarding authenticity and reflexiv- 
ity helps researchers move deeper into the investigation and captures participants’ voices 
and empowers them to a greater extent. 

Results 

The results section pertaining to Part I comprises Step 8 (collect data) and Step 9 (analyze 
data), and as noted by Leech and Onwuegbuzie (2010), the results section includes the 
ethical nature of data collection, research paradigm, and philosophical correlates of the 
research paradigm. Qualitative research has been described as rich in multiple meanings 
(Onwuegbuzie & Leech, 2007a), and also maintains a communitarian view of power, as 
described by Christians (2005), as being intimate and reciprocal. The results section of 
qualitative research involves interpretive validity, or the extent to which the interpretation 
of the analysis represented an understanding of the phenomenon (Maxwell, 1992), 
through thick and rich descriptions using original language from the participants for each 
theme and category. Writing the qualitative research report involves aspects such as the 
clear description of themes, appropriate displays of meanings of themes, and presenting 
thick, rich data (Onwuegbuzie & Leech, 2010). Threats to verification, trustworthiness, 
internal credibility, and authenticity are addressed through some of the relevant queries as 
outlined by Miles and Hubennan (1994): (a) How context-rich, meaningful, and thick are 
descriptions from text? and (b) Are the presented data linked to the emerging theory? 
The concept of truth space (the sample of words portrays the experience, the feelings, the 
opinions, the sentiments; Onwuegbuzie, 2003) should be regarded in the results of a 
study whereby “qualitative researchers hope that the sample of words is representative” 
(Onwuegbuzie, 2003, p. 400). Hence, students address components throughout the results 
section, as they also strive to integrate data and avoid vague, ambiguous, and emotional¬ 
laden tenns and represent the phenomenon studied. 

Discussion 
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According to Leech and Onwuegbuzie (2010), steps of the Discussion section comprise 
the legitimation of data (Step 10) and the interpretation of data (Step 11), and conclude 
with a written report (Step 12), written in a manner acceptable to the intended audience 
(i.e., faculty, journal reviewers, policy makers, practicing educators in the field, confer¬ 
ence paper reviewers, researchers). With respect to each tradition of inquiry (e.g., biogra¬ 
phy, phenomenological, grounded theory, ethnography, case study), two rhetorical struc¬ 
tures are addressed: (a) the overall structure, or the overall organization of the study; and 
(b) the embedded structure, or the specific narrative devices and techniques used by a 
writer (Creswell, 2007). Furthermore, the discussion section allows students to interpret 
findings and reference these findings back to research questions. As a result, the discus¬ 
sion section of the Checklist also allows researchers to reevaluate research questions, and 
if necessary, to refonnulate research questions (Step 13) as a result of data interpretations 
(Leech & Onwuegbuzie, 2010). The Checklist further presents safeguards for students to 
understand personal bias, generalizations, and implications. 

Reference List and Appendix 

The final two sections of Part I of the Checklist, namely, the reference list and appendix, 
direct students to attend to important features such as accuracy of citations (e.g., citations 
that are provided in the text are also contained in the reference list), and accuracy of APA 
(2010) style guidelines when referencing authors. Other items in this section include: (a) 
names of all authors are accurate and consistent, and (b) sources are written accurately. 
The appendix section of the Checklist refers to eight critical elements for an appendix, 
including appropriate qualitative software output, informed consent documentation, and 
samples of any instruments developed or adapted by the researcher. 

APA Style: Part II of the Checklist 

Chapter 1 of the Publication Manual of the APA (2010) pertains to writing for the behav¬ 
ioral and social sciences, and describes several considerations for authors, specifically 
regarding their own research and the scientific publishing tradition. According to Henson 
(2007), more than 60% of educational journals requested that authors adhered to APA 
(2010) style guidelines. Also, Onwuegbuzie and Combs (2009) contended that authors 
who write with discipline and strict adherence to style guidelines likely will increase pub¬ 
lication chances, and also increase the likelihood to publish in a journal with high visibil¬ 
ity for stakeholders and policymakers. Designed to address integrating writing style ele¬ 
ments in coursework, Part II of the Checklist is presented in the same organizational for¬ 
mat as Part I, and is the scoring rubric for writing quality and adherence to APA (2010) 
style guidelines. In addition, the final page of Part II of the Checklist allows the instructor 
to add additional points for a student who has provided detailed references with respect to 
one or more sections of a notebook. Thus, Part II of the Checklist facilitates the technical 
application of writing style for students through the exemplar qualitative notebooks. Ap¬ 
pendix A presents Part II of the Checklist (i.e., adherence to APA [2010] style guide¬ 
lines), directly after Part I of the Checklist. 
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The Interactive Writing and Feedback Process 

According to Richardson and St. Pierre (2005), writing represents a method of inquiry 
and is more than a reactive process. In the interactive writing and feedback process of the 
qualitative course model for teaching and learning qualitative research (Onwuegbuzie et 
ah, 2009), writing is developmental, interactive, and a meaning-making endeavor. The 
Checklist Part I and Part II, with the Notebook assignments, facilitate dialogue between 
students and instructors toward the exchange of ideas and collaboration for writing re¬ 
ports. Moreover, students are led to recognize that the Checklist is a foundational guide 
for qualitative research, and that becoming a life-long qualitative researcher is indeed 
possible. However, both instructors and students must maintain particular roles and re¬ 
sponsibilities during the interactive writing and feedback process so that data collection, 
data analysis, and data interpretation might be integrated and internalized for autonomous 
learning. 

Instructor Role and Responsibilities 

Feedback is considered a strong element for effective learning (Bangert-Drowns, Kulik, 
Kulik, & Morgan, 1991). Detailed feedback is at the crux of an instructor’s responsibility 
as a professional educator, researcher, and motivator. As such, formative feedback helps 
students reflect on their levels of performance and take greater responsibility for their ac- 
tions (Butler, 2004). Appendix B presents an excerpt from one qualitative notebook and 
the detailed feedback given by an instructor using the comment feature of a Word docu¬ 
ment. 

Student Role and Responsibilities 

Students who are active participants in learning are more likely to understand course ma¬ 
terial than if they are passive receptors in a classroom presentation (Ramsden, 1992). Fur¬ 
thermore, Le Brun and Johnstone (1994) suggested that student attitudes and values are 
affected by class participation, and that participation also increases motivation and em¬ 
phasizes a student’s responsibility for learning. Students’ responsibilities include address¬ 
ing feedback presented by instructors in a reflective and meaning-making endeavor to 
include attending (spending time with the event) and affirming (accepting and valuing the 
event, even if the first response may be to disregard it) (Kolb, 1984). Thus, it is important 
to note that students should regard feedback as a positive, developmental component of 
writing qualitative reports. 


The Reflexive Journal 

Systematic reflexivity is an integral consideration in qualitative research, and “collecting 
and critically reflecting on researcher bias in a systematic manner can greatly enhance the 
legitimation of accounts of social and behavioral phenomena” (Onwuegbuzie et ah, 2008, 
p. 2). Through the reflexive journal (Cunliffe, 2004), students regard all aspects of the 


3 Appendix B is posted online at http://www.uncw.edu/cte/et/articlesWoll 1 1/FrelsAppB.pdf . 
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qualitative research course, including the qualitative notebooks, the biases they bring to 
the course, their personal investment in and commitment to the course, and their relation¬ 
ships as a result of the interactive writing and feedback process. As a result, the purpose 
of the reflexive journal is two-fold: (a) to become a critically reflexive practitioner 
(Cunlife, 2004) and (b) to integrate reflective learning (Kolb, 1984). Thus, students main¬ 
tain the journal through personal entries on a weekly basis. 

Reflections on the Process 


Students ’ Perspectives 

Upon first impression, students who have not taken a research class might be over¬ 
whelmed at the extensive nature of the Checklist. Even though the terminology of the 
Checklist items is introduced in Phase I and Phase II of the qualitative course, the list 
might appear intricate or complicated upon first glance. To understand the cognitive and 
emotional responses of students taking the four phase qualitative course, Frels, Stark, and 
Onwuegbuzie (2010) examined the reflexive journals of one class and uncovered that 
emotional changes accompanied the developmental phases of learning. With respect to 
the initial introduction to the Checklist, one student noted, 

Oh my goodness! We got a rubric tonight about our first notebook and I had no 
idea of any of the words the professor used or even what he was talking about! It 
is a good system, but sometimes we rush through so much I tend to zone. I asked 
and he clarified and then I think I understood a little bit more, but I feel like a 
contestant on the Amazing Race. I get a clue and then use my energy and detec¬ 
tive skills to figure out what’s going on. I think I’m smart enough, and I think I 
can persevere, so okay. But, I really love the enthusiasm that our teachers have. 

Another student explained, 

The class work is very demanding. I am not sure what to expect on the first write 
up. There were a lot of challenges. I could not believe the immense number of 
items on the rubric. If I can make a suggestion... it would be the need for a model 
write up. 

In addition, as students receive feedback via both Part I and Part II of the Checklist at the 
same time, instructors might recognize that even though fonnative feedback is a purpose 
of the Checklist the first time students receive the Checklist, they might focus more on 
their number score than their developmental growth as a researcher. One student ex¬ 
plained, 

I think I did well on my first qualitative notebook, but I am still not sure what my 
grade is. I earned a B on one section, a C on another section, and an A at the end. 
I hope the A is correct; it was on the last page. 

Similarly, when receiving the first assignment’s feedback, another student noted, 
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The rubric is very confusing, and I believe that the professors should have taken a 
class session to explain the points. There were many times that I thought I had 
mentioned the appropriate information, but I was informed that I stated it incor¬ 
rectly. 

Taking into account the detailed nature of the Checklist, the emotional response of stu¬ 
dents might include feelings of being overwhelmed due to not only the new concepts in¬ 
troduced, but also to the vast number of items included in the scoring checklist. Indeed, 
one student remarked, 

I feel lost at times when I am writing in this class because of the strict rubric. I sat 
down and went through each point and found that this is going to be very difficult. 
Some of the material that we have to cite is nowhere to be found. I mean it is 
tough, and writing for such rigorous standards for the first time is a challenge. I 
hope that our inexperience is taken into account. It should be an interesting write 
up. It will take time to master this writing style. With practice anything can be 
done. 

Furthermore, course instructors should recognize that even though the scoring checklist 
provides the framework for students to write each qualitative notebook, it is the feedback 
that accompanies the assignment that promotes reflective growth. Often times, students 
might not respond to the feedback positively. In fact, one student responded, “We got 
back our first papers and I was a bit embarrassed with the errors I made regarding cita¬ 
tions. I could have made a decent A without them.” However, to evidence the confidence 
that accompanies the third and fourth phases of the course, one student noted that, 

I arrived 30 minutes early tonight and mapped out our last three assignments. I 
really think I would like to do qualitative research, but just not for my disserta¬ 
tion; I want to do it in my career. I have been thinking so much this week about 
how to even write the paper on ethnographic analysis and discourse analysis, be¬ 
cause truly I can only skim the surface with these designs...I wish we had more 
time. This class is too short. 

Also, with respect to the writing growth that occurred after the use of the Checklist, an¬ 
other student observed, 

The [Checklist] has begun making sense. I met with the professors and found that 
they can explain concerns at a human level. I was impressed with the effort taken 
to design such a rubric. Honestly, once I began to believe in it, it made a lot more 
sense, and I could begin to see how I needed to address these issues in my own 
write-ups. When I wrote my first write-up it became apparent to me that I was not 
a competent writer. 

As a testament to the impact of the use of the Checklist, three of the seven doctoral stu¬ 
dents participating in the four-phase model qualitative research course deemed the 
Checklist as foundational in the success of their dissertation research. Other students in 
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the class either engaged in quantitative dissertation studies or have yet to complete the 
dissertation process. In fact, at the conclusion of the class and in the emergent scholar 
phase, the three doctoral students who used the checklist to help in conducting their dis¬ 
sertation research embarked on co-authoring this article because they believed that their 
qualitative expertise and writing expertise were results of the use of the Checklist as both 
a guide and an assessment tool. 

Instructors 9 Perspectives 

As noted by Frels et al. (2010), professors of the four-phase model for teaching and learn¬ 
ing qualitative research observed that teaching this course for the first time at an institu¬ 
tion might generate heightened levels of anxiety initially because students are taken on a 
journey with no prior history as to how this journey will end. However, as the course un¬ 
folds and students advance down the path of learning, many, if not most, of them begin to 
trust that the destination will yield positive outcomes. The second time the course is 
taught at the same institution, one or more students who previously took this course are 
asked to team-teach the course, thereby serving as teaching assistants. These teaching as¬ 
sistants not only can validate the initial trepidation of the new students, but also they can 
assure them that the journey on which they are embarking will maximally prepare them 
to be both consumers and producers of qualitative research. Thus, these teaching assis¬ 
tants continue to grow as emergent scholars (i.e., Phase 4), as they help the new students 
negotiate the first three phases of the course. Indeed, these teaching assistants likely are 
in a better position than are the instructors to identify the items on the Checklist that are 
causing the most problems. In addition, the teaching assistants facilitate the new students’ 
understanding of the interactive nature of the course (e.g., how the notebook, instructor 
feedback, and reflexive journal processes work together), which is key for the new stu¬ 
dents’ journeys. As this course is taught over the years, a culture of scholarship ensues. 

Implications 

A qualitative course that combines experiential learning through a dialogical process al¬ 
lows participants, as noted by Brookfield (1985), to “encounter experiences, attitudinal 
sets, differing ways of looking at their personal, professional, political, and recreational 
worlds and a multitude of varying purposes, orientations, and expectations” (p. 41). The 
changing field of qualitative research, as noted by House (2005), involves the role of re¬ 
searcher as a methodological and interpretive bricoleur acting as “an artist, a quilt maker, 
a skilled craftsperson, a maker of montages and collages [who can] interview, observe, 
study material culture, think within and beyond visual methods...” (p. 1084). In her ap¬ 
proach to teaching qualitative research, Lapadat (2009) explained a “triumvirate of meth¬ 
odology instruction” (p. 957) wherein learning involves three main strands: (a) learning 
what (i.e., familiarity with the language, concepts, theories, history, debates within the 
field); (b) learning how (i.e., focusing on a study, conducting interviews, transcribing 
tapes); and (c) recognizing doing (i.e., engaging and identifying with the seifs values, 
emotions, and understanding the human conditions associated with qualitative research). 
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In addition, Bizzell (1992) explained that often times instructors fall into the trap of 
teaching academic discourse “as if it were a matter of a static relationship between indi¬ 
vidual student and ‘body’ of knowledge: step by step, through ‘requirements’ and ‘pre¬ 
requisites’” (p. 145). As a result, this approach might leave students struggling to seek 
comprehensive knowledge to increase understanding (Bizzell, 1992). Conversely, the 
Checklist serves as a guide for technical and cognitive discourse because it seems to be a 
repetitious and stabilizing component to increase learning and dialogue. 

Even though the interactive use of the Checklist and the Qualitative Notebooks was de¬ 
signed for the purpose of an introduction to qualitative research course involving data 
analysis for a case study, the Checklist would be a valuable tool as both a guide and ru¬ 
bric for other frameworks within the qualitative tradition (e.g., phenomenological re¬ 
search, ethnographic research, grounded theory). In addition, items could be added or 
subtracted from the Checklist to adapt it for various levels of instruction (e.g., under¬ 
graduate, master’s level). Thus, the Checklist becomes a launching pad for students and 
beginning researchers to embark upon the technical, logistical, ethical, personal growth, 
and awareness involved in becoming emergent qualitative scholars. 

Conclusion 

Knowledge is acquired through an individual’s interaction with social processes and con¬ 
texts (Kelly, 1955; Piaget, 1954; Vygotsky, 1978). Kolb (1984) contended that learning is 
a continuous, holistic, and adaptive process wherein a person experiences a range of emo¬ 
tions, increased awareness, and innovative conceptualizations. Hence, the interaction of 
writing and feedback (socially and contextually) between instructors and students after 
using the Checklist continues well after students have completed the qualitative course- 
work. Describing a holistic approach to academic writing, educators Antoniou and 
Moriarty (2008) noted that “privately, many academics struggle with their writing, that 
they often cannot find time and space to write, experience fear and anxiety about writing, 
feel they lack knowledge and expertise, and worry they are not good enough” (p. 158). 
With this in mind, as instructors and students, we hope that the Checklist, when used with 
the 13-step methodological framework (Leech & Onwuegbuzie, 2010) for qualitative re¬ 
search, offers advantages for instructors and sheds light upon the qualitative research 
process for students for confidence in writing rigorous qualitative research reports. 
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